1. Technical Field
The invention is related to a system and method for optimizing a graphics intensive software program for a user""s computer graphics hardware. Examples of such graphics-intensive programs include flight simulations, computer aided design programs, photo-editing programs, computer games, and clear type and presentation software.
2. Background Art
Due to advances in graphical operating systems and 3D computing, there has been an enormous increase both in how much data is sent to computer monitors and the sophisticated calculations that must be done to determine what is seen on the screen. As a result, computer video cards have evolved to be much more like co-processors. The video card in a personal computer plays a significant role in the following important aspects of the personal computer system:
Performance: The video card is one of the components that has an impact on system performance. For example, application software that depends on a high frame rate (how many times per second the screen is updated with new information) for smooth animation, is often impacted far more by the choice of video card than by the choice of system Central Processing Unit (CPU).
Software Support: Certain programs, such as, in particular, video games and other graphics programs, require support from the video card. Some programs, such as 3D-enhanced games, will not run at all on a video card that does not support them.
Reliability and Stability: Choosing the wrong video card can cause problematic system behavior, such as computer system crashes and hangs. In particular, some cards or types of cards are notorious for having unstable drivers, which can cause a host of difficulties.
Because the computer screen is two-dimensional (2D), everything that a personal computer displays must be two-dimensional as well. In order for the computer monitor to display three-dimensional (3D) objects, it is necessary for them to be converted to 2D images. This requires special processing and a large amount of computing power. The push for more realism, more detailed graphics, and faster speeds in such programs as action games, flight simulators, graphics programs, computer aided design (CAD) applications and presentation software, means that more 3D work must be done in a shorter period of time, requiring the use of specialized 3D accelerators. To perform the large amount of computation work necessary to translate 3D images to 2D in a realistic manner, accelerators were designed that did much of this work with specialized hardware, instead of forcing the system processor to do it. Using a 3D accelerator allows programs to display virtual 3D worlds or 3D objects with a level of detail and color that is impossible with a standard 2D video card. Common 3D operations wherein 3D accelerators are used include:
Gourad Shading: This is an algorithm that is used to give 3D surfaces realistic shading. The effect helps the object appear to have depth and helps to define the shape better.
Clipping: This operation determines what part of an object is visible on the screen and xe2x80x9cclips outxe2x80x9d any part that the user cannot see. Parts of objects that are off-screen are ignored, thereby improving system performance because the off-screen portions do not have to be computed for rendering.
Lighting: Objects in the real world have their appearance shaped by the light sources in the scene. Lighting effects cause color shading, light reflection, shadows and other effects to be added to objects based on their position and the position of light sources in the room.
Transparency/Opacity: Objects in the real world are transparent, semi-transparent or opaque. These transparency/opacity features are emulated in software/hardware.
Texture Mapping: For realistic objects, it is necessary to overlay pictures on them to give them texture. Texture mapping allows objects to be made so that they appear to have substance instead of being xe2x80x9cflatxe2x80x9d. There are several different types of texture mapping that are used by various software and hardware.
Dithering: This is an effect that is actually used in many different places, including regular 2D graphics and also in printing. Dithering is the process of mixing a small number of colors together in specific patterns to create the illusion of there being a larger number of colors. In 3D, it is used largely to show more realistic color without needing to increase the color depth of the image (which means more computation time and more memory to store the graphics). Dithering takes advantage of the eye""s tendency to blur spots of different colors by averaging their effects and merging them into a single perceived shade or color.
Fogging: Fogging serves two purposes by blurring objects that are in the distance. First, it helps to make the scene appear more realistic. Second, fogging allows the 3D process to be performed more quickly because those objects in the distance that are xe2x80x9cfogged outxe2x80x9d can be computed more quickly since they are shown in less detail.
Filtering: There are several types of filtering that can be applied to an image. These are used to xe2x80x9cclean upxe2x80x9d the image and smooth out textures and shapes. In particular, bilinear filtering is used when showing textures up close to remove the xe2x80x9cblockyxe2x80x9d look that results from magnifying an object when showing it at the front of a scene. Bilinear filtering refers to a process where the graphics hardware smooths the appearance of a texture by interpolating from one pixel value to the next. Textures appear perfectly smooth, even if viewed from very close, thanks to bilinear filtering.
In order to benefit from 3D acceleration features, it is necessary for software to know about them and support them. Unfortunately, with 3D being a relatively new field, a whole new breed of different and incompatible video cards has hit the market. Different cards have different capabilities. Additionally, different cards will behave differentlyxe2x80x94that is, the same instructions given to various video cards can yield different results. Support for a particular card is required from a program if it is to take advantage of the 3D card""s features. Otherwise, the program will not benefit much (or at all) from the 3D hardware. Most of the specific support for 3D hardware is from games. This comes usually in the form of special versions that have been tailored to different video cards. In addition to a regular version of a popular game, a version may be created by a company to support a particular 3D chipset. This version will usually either have much superior graphics, faster performance, or both. However, it typically will not work with a different type of 3D card. Fortunately, new standard libraries are being developed to tackle this problem. Driver libraries like Microsoft Corporation""s Direct3D(copyright) and OpenGL are designed to allow software to be written generically without tailoring them to each 3D chipset on the market, allowing them to be used regardless of what chipset is used. Software applications that are designed to use these libraries can avoid some of the need for customization.
Many applications employ 3D graphics and 3D accelerators, as well as driver libraries such as Direct3D(copyright), to improve the performance and reliability of the application software. One such application, for example, is presentation software, such as Microsoft Corporation""s PowerPoint(copyright). PowerPoint(copyright) has a capability of providing a xe2x80x9cslideshowxe2x80x9d feature wherein a presenter can present a presentation in slide form to an audience using a computer.
Customers desire great performance in slideshow when they present. They want the animations to look as smooth as the ones they see on television. In order to do this, the PowerPoint(copyright) presentation program uses hardware graphics acceleration provided by the Direct3D(copyright) driver library. Unfortunately, because some video cards support certain graphics features, while others do not, when the application software tries to use them, they do not work properly. For example, some cards crash or hang on certain function calls. Obviously, it is extremely important for the slideshow feature to work reliably. If a game crashes, the user can reboot and start again without much issue. However, if a slideshow crashes it is another matter. For example, if a salesperson is giving a presentation to a large group of people and the slideshow crashes, the audience may consider the presenter incompetent.
The present invention overcomes the aforementioned limitations in prior computer software and hardware systems by a system and method that seeks to get the best software application performance and reliability by optimizing the application software, such as PowerPoint(copyright), for the capabilities of specific video hardware. This system and method encompasses a procedure that validates the different functions of a 3D acceleration capable video card, decides whether to use the acceleration hardware and optimizes the software application to selectively use the functions that work on the specific video acceleration card. This procedure is typically run every time just as the application software initializes. The invention can be couched in the terms of use with a graphics presentation program, such as PowerPoint(copyright), and a graphics accelerator driver database such as Direct3D(copyright). However, those with ordinary skill in the art will realize that the system and method according to the present invention could be used with any graphics intensive application program that employs the use of graphics acceleration hardware in combination with any graphics driver library.
By way of example, a software application, such as PowerPoint(copyright), typically interfaces with a 3D graphics accelerator via a driver library such as Direct3D(copyright) to more efficiently process graphics displays. In the case of the PowerPoint(copyright) software application, a slideshow module is included which presents the xe2x80x9cslidesxe2x80x9d in the user""s presentation as a cohesive package. This slideshow module interfaces with a drawing layer that provides drawing commands to display these slides on a computer monitor or other display device. The drawing layer can interface with the Central Processing Unit (CPU) of the user""s computer either via a software renderer to provide the drawing commands, or can interface with a graphics accelerator renderer, such as the Direct3D(copyright) renderer, which interfaces with the graphics accelerator to draw the respective drawings.
The graphics accelerator renderer, which typically resides in the application software, interfaces with the graphics accelerator hardware and software through a drawer, which converts drawing layer commands to graphics accelerator commands (in this case Direct3D(copyright) commands). Additionally, the graphics accelerator renderer initializes the graphics accelerator hardware and software via an initialization module for compatibility testing of the graphics acceleration functions and determining which functions are to be utilized with the application software. The present invention automatically performs compatibility testing with the graphics acceleration hardware and software as part of the initialization process. It should be noted that it is typically desirable to use the graphics accelerator hardware rather than the software with this invention, since this hardware processes 3D data much more quickly than either the software resident in the graphics accelerator or the software renderer that interfaces with the system""s CPU. The test module also interfaces with a drawer and texture manager for the purpose of performing the aforementioned compatibility testing of the graphics acceleration hardware of the video card.
In the most general sense, the invention performs compatibility testing of the graphics hardware by first checking to see if graphics acceleration is selected or enabled, checking if the video card is on a list of cards with known problems, and verifying that a graphics acceleration driver library initializes the graphics accelerator successfully. The invention also checks to see if sufficient video memory is available to perform the various compatibility tests and verifies that calls to the video card hardware are successful. In each of these process actions if the particular process action indicates that there is a problem with the compatibility of the video card and the application software the decision is made not to use the graphics acceleration hardware of the video card. Additionally, if the above process actions indicate no compatibility problems exist, the options to be used with the video card are selected and stored. Then sub-pixel positioning is tested. If this test is successful, then pixel offset values are stored. If the test is not successful, the decision is made not to use graphics acceleration hardware of the video card. A test is then conducted to determine whether the opacity function performs correctly when used with the graphics accelerator. Again, if the opacity function does not work correctly then the procedure is exited with a determination not to use graphics acceleration. If the opacity test is passed, however, a test is conducted for color replacement. If this test does not pass, another test wherein a fog function is used to perform color replacement is conducted, and if this test also does not pass then the decision is again made not to use graphics acceleration. If the color replacement test passes, then the decision is made to use graphics acceleration, and the system and method according to the present invention continues to run the application software. Those with ordinary skill in the art will realize that it is not necessary to perform all of the above tests in a specific order. Additionally, other types of tests could be performed to ensure software application and video card hardware compatibility before the software application is used to render 3D graphics.
By way of example, the software program, in this case PowerPoint(copyright), typically initially examines whether the user has turned off the hardware graphics acceleration option. To this end, a graphical user interface may be made available to the user to disable the graphics acceleration option. If the user chooses to disable the graphics acceleration option, a flag is set in the software. The application software may also set this flag if a previous attempt to graphics acceleration fails. The application software may then examine whether this flag is set in the software. If so, the system and method according to the present invention determines that the graphics acceleration hardware of the video card is not to be used, and exits the procedure. If the user or software has not turned off the hardware graphics acceleration option of the software, the system continues to examine the feasibility of using the graphics acceleration hardware of the video card.
In one embodiment, the software application then examines whether the video card resident in the user""s computer hardware is on a list of cards that is known to be incompatible with the software program. This may be performed by the application software by maintaining a database of video cards and/or video drivers known to be incompatible with the software and checking to see if the video card and/or video driver resident in the user""s computer is in the database. If the card and/or driver is determined to be incompatible with the application software by this action, the application software does not use the graphics acceleration hardware, and exits the procedure.
The application software then attempts to initialize the graphics acceleration hardware by setting up the drawer in the application software, ensuring it loads successfully and making calls to it. If the initialization is unsuccessful, the system determines that the graphics acceleration hardware is not to be used and exits the procedure. If the initialization is successful, the video memory of on-screen and off-screen buffers of the video card is then checked by requesting the video memory for them. If this call is unsuccessful the decision is made not to use the graphics acceleration hardware. If the call is successful then the system continues its checks.
The system then requests to access the video card hardware through the hardware abstraction layer (HAL). The HAL is a software interface to various 3D features of the video card. If the hardware device cannot be accessed, the decision is made not to use the graphics acceleration hardware. If access to the graphics acceleration hardware is successful, the system and method according to the present invention proceeds to set the graphics acceleration options necessary to support the application software. These options are saved in memory and accessed as the application software runs. Such options include, for example, how textures are to be drawn and the selection of various filtering options.
The system then checks if there is sufficient video memory to perform the remainder of the compatibility tests between the application software and the graphics hardware. During these tests, test patterns and computations are written to off-screen video memory. Hence, both the on-screen and off-screen buffers are used. To draw the test patterns and to perform the applicable calculations of the present invention, there typically must be enough video memory for the on-screen and off-screen buffers plus the number of textures used in the compatibility tests (in a tested embodiment this was 8 textures of size 128xc3x97128 pixels).
The system then creates a texture, filling it with a test pattern. The test pattern preferably consists of a black background and diagonal line of pixels of all white color except the first three pixels which are red, green and blue. This test pattern is typically drawn to the off-screen buffer at position 1,1.
Next the sub-pixel positioning is tested. Sub-pixel positioning is important to presentation software, such as PowerPoint(copyright), because the clarity of text on the computer screen is critical to the readability of the presentation. In general, this sub-pixel positioning test consists of 1) drawing the test image to off-screen video memory; 2) determining where the pixels for the test image were drawn using the expected pixel boundary and color; 3) calculating the offset values between where the pixels were expected to be drawn and where they were actually drawn; and 4) determining whether the offset values are acceptable to correct for this offset during 3D graphics rendering. If the offset values are not acceptable, different offset values may be tried until the offset values are below some predetermined threshold. This process may be repeated for more than one cycle.
Specifically, for the test conducted by the system and method according to the present invention, the test pattern is preferably drawn to off-screen video memory and it is assumed that the center of a test pixel is at 0.5, 0.5 (further assuming a pixel matrix of 128xc3x97128 pixels and having an origin of 0,0). The pixel values of the test pixel are then read back from the off-screen video memory. If the values match what is expected (i.e, the values are below a given threshold) then this sub-pixel positioning is used (stored in memory to correct for pixel position during the application run time). That is, if the difference in colors from what is expected and what is read from the screen buffer is less than the threshold, then this sub-pixel positioning is used. One exemplary formula that may be used for this purpose is (abs(r_actualxe2x88x92r_expected)+abs(g_actualxe2x88x92g_expected)+abs(b_actualxe2x88x92b_expected)), where r, g and b represent the pixel colors of red, green and blue, respectively. For example, using a threshold of 24, if a RGB color of (255,0,0) is expected and an actual color of (250,1,0) is read, then the difference in colors between the actual and expected values is 6 so this sub-pixel positioning is accepted. If the read values do not match (i.e., the values are over the threshold), then other pixel offset values are tried in an attempt to determine the actual pixel offset until a match is found. This process may be repeated for a number of cycles or iterations. In a tested embodiment four cycles were completed, using expected pixel offset values of 0.5+{fraction (1/16)}, 0, 0+{fraction (1/16)} and 0.5 from the origin of each pixel center as drawn to offscreen video memory by the graphics hardware.
If the difference in value between the expected pixel color and the read pixel colors is greater than the threshold (i.e., 24) for all desired cycles using different offset values, then the minimization filter and maximization filters are turned off and the desired number (e.g., four) of cycles of the above process is repeated again. The maximization filter is used when an image is being increased in size. It uses interpolation between pixels so the image looks smooth (usually bilinear filtering). The minimization filter is used when the image is decreased in size. It also uses interpolation but typically uses a different algorithm than the maximization filter. If turning off these filters still does not bring the difference between the expected pixel color and the read pixel values under the threshold then the determination is made not to use graphics acceleration as the resultant sub-pixel positioning will result in noticeable distortion or blurring of the 3D image.
If the sub-pixel positioning is determined to be suitable, the system and method according to the present invention then tests to see if the graphics acceleration hardware can perform the opacity function. This is done by setting the opacity of a pixel to a prescribed value such as 50%. The color value of the pixel is then read from the screen buffer. If the color has not changed then it is assumed that the card cannot perform the opacity function and graphics acceleration is not used.
The color replacement function is then tested. Several conventional 3D acceleration techniques can be used for this purpose. One of these techniques tests color replacement by replacing all of the color in a texture by another color without changing the opacity values. For example, a texture is drawn in red by telling the video card to use Diffuse Color for the texture and to ignore the original color of the texture. The color of the pixels is then read. If the color is not as expected then an attempt is made to determine whether a fog function can be used to mimic the opacity function. If this fails, then graphics acceleration function is not used. However, if all of the above steps were successful then the hardware graphics acceleration function is used.
Even with the above-specified compatibility tests, there is a chance that the hardware will fail. To mitigate this possibility, the present invention automatically disables graphics acceleration if the graphics accelerator hardware crashes or hangs during the running of the software application. The next time the user launches application software, slideshow in the case of PowerPoint(copyright) application software, it does not crash. To do this, a flag is set when slideshow is started and unset it once the slideshow successfully ends. However, if slideshow does not successfully end then the flag is not unset. The next time slideshow is launched, the program notes that the flag is set and graphics acceleration is not used.